Papers with text normalization
Dive into Deep Learning for Natural Language Processing (D19-2)
Copied to clipboard
| Challenge: | GluonNLP is a powerful new toolkit that automates the most laborious aspects of deep learning for NLP. |
| Approach: | This hands-on tutorial demonstrates how to scale unsupervised pre-training techniques with Apache MXNet and GluonNLP. |
| Outcome: | This hands-on tutorial examines the challenges of scaling these models and algorithms effectively with Apache MXNet and GluonNLP. |
PolyNorm: Few-Shot LLM-Based Text Normalization for Text-to-Speech (2025.emnlp-industry)
Copied to clipboard
| Challenge: | Text Normalization (TN) is a key preprocessing step in Text-to-Speech systems. |
| Approach: | They propose a prompt-based approach to TN using Large Language Models (LLMs) they propose scalable experimentation across languages to reduce the reliance on manual rules . |
| Outcome: | The proposed approach reduces the reliance on manual rules and enables broader linguistic applicability with minimal human intervention across eight languages. |
hinglishNorm - A Corpus of Hindi-English Code Mixed Sentences for Text Normalization (2020.coling-industry)
Copied to clipboard
| Challenge: | hinglishNorm is a human annotated corpus of Hindi-English code-mixed sentences for text normalization task. |
| Approach: | They propose to annotate sentences in Hindi-English code-mixed sentences using a human annotated normalized form. |
| Outcome: | The proposed corpus contains 13494 segments annotated for text normalization. |
Neural Text Normalization with Subword Units (N19-2)
Copied to clipboard
| Challenge: | Text normalization (TN) is an important step in conversational systems. |
| Approach: | They frame text normalization as a machine translation task and tackle it with sequence-to-sequence models. |
| Outcome: | The proposed model normalizes written text to its spoken form to facilitate speech recognition and text-to-speech synthesis. |
Adversarial Text Normalization (2022.naacl-industry)
Copied to clipboard
| Challenge: | Text-based adversarial attacks are becoming more commonplace and accessible to general internet users. |
| Approach: | They propose a method that restores baseline performance on attacked content with low computational overhead. |
| Outcome: | The proposed method restores baseline performance on attacked content with low computational overhead. |
Benefits of Data Augmentation for NMT-based Text Normalization of User-Generated Content (D19-55)
Copied to clipboard
| Challenge: | Social media texts are considered important language resources for several NLP tasks, but their use of non-standard words makes it difficult to process and analyze UGC. |
| Approach: | They propose to use a Neural Machine Translation approach to normalize lexical variants to their canonical forms to overcome performance drop in UGC. |
| Outcome: | The proposed approach overcomes a data bottleneck in Dutch, a low-resource language. |
SpeechWeave: Diverse Multilingual Synthetic Text & Audio Data Generation Pipeline for Training Text to Speech Models (2025.acl-industry)
Copied to clipboard
| Challenge: | Text-to-Speech (TTS) training requires extensive and diverse text and speech data. |
| Approach: | They propose a synthetic speech data generation pipeline that generates multilingual, domain-specific datasets for TTS training. |
| Outcome: | The proposed pipeline generates data that is 10–48% more diverse than baseline across various linguistic and phonetic metrics, along with speaker-standardized speech audio while generating approximately 97% correctly normalized text. |
Utilizing Character and Word Embeddings for Text Normalization with Sequence-to-Sequence Models (D18-1)
Copied to clipboard
| Challenge: | Recent advances in text normalization have limited applications in other languages . a novel approach to text normalizing uses character embeddings and word embedds . |
| Approach: | They propose a sequence-to-sequence model with character-based attention that uses pre-trained word embeddings to model subword information. |
| Outcome: | The proposed model achieves state-of-the-art F1 score on Arabic spelling correction task despite being small and unsuited for the task. |
Fast and Accurate Reordering with ITG Transition RNN (C18-1)
Copied to clipboard
| Challenge: | Attention-based sequence-to-sequence neural networks learn to jointly align and translate. |
| Approach: | They propose to use a reordering RNN that shares the input encoder with the decoder to decouple re-ordering from translation. |
| Outcome: | The proposed model can achieve superior reordering accuracy without feature engineering and is 2.5x faster in decoding. |
Service-oriented Text-to-SQL Parsing (2020.findings-emnlp)
Copied to clipboard
| Challenge: | TEXT2SQL models focus on generating complex SQL query in a precise and complete fashion . current models focus only on executing complex queries in production environment . TExT2sql models are limited by human labor and limited by predefined template . |
| Approach: | They propose a text-to-sql parser that translates natural language utterance to SQL query . the framework enables flexibly access database than rigid API in the application . |
| Outcome: | The proposed model outperforms baseline models in the WikiSQL task by 13% error reduction. |
SciCap: Generating Captions for Scientific Figures (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Low-quality captions are common in scientific articles and can decrease understanding . this paper aims to develop an end-to-end neural framework to generate informative, high-quality figure captions for scientific figures and charts. |
| Approach: | They propose an end-to-end neural framework to automatically generate captions for scientific figures from a large-scale dataset . they used figure-type classification, sub-figure identification, text normalization, and caption text selection to build models that caption graph plots, the dominant figure type. |
| Outcome: | The proposed model can generate high-quality captions for scientific figures and charts from a large figure-caption dataset from arXiv. |
TNT: Text Normalization based Pre-training of Transformers for Content Moderation (2020.emnlp-main)
Copied to clipboard
| Challenge: | Language model pre-training (self-supervised or unsupervised learning) has been widely used in a multitude of language processing tasks such as named entity recognition, sentiment analysis, question answering and content moderation. |
| Approach: | They propose a new language pre-training model TNT for content moderation that uses a combination of masking strategy and text normalization to learn from text. |
| Outcome: | The proposed model outperforms baselines on hate speech classification task and is a potential approach to misspelling correction. |
Seq2Edits: Sequence Transduction Using Span-level Edit Operations (2020.emnlp-main)
Copied to clipboard
| Challenge: | Seq2Edits is an open-vocabulary approach to sequence editing for natural language processing tasks with a high degree of overlap between input and output texts. |
| Approach: | They propose an open-vocabulary approach to sequence editing for NLP tasks with a high degree of overlap between input and output texts. |
| Outcome: | The proposed approach speeds up inference by up to 5.2x compared to full sequence models . it improves explainability by associating each edit operation with a human-readable tag. |
Open ASR for Icelandic: Resources and a Baseline System (L18-1)
Copied to clipboard
| Challenge: | Existing language resources are not sufficient for less-resourced languages, but a system with sufficient resources is needed. |
| Approach: | They describe available language resources and their preparation for use in a large vocabulary speech recognition system for Icelandic. |
| Outcome: | The proposed system improves on acoustic training sets and a speech corpus with a pronunciation dictionary. |
A Spelling Correction Corpus for Multiple Arabic Dialects (2020.lrec-1)
Copied to clipboard
| Challenge: | Arabic dialects are non-standard varieties of Arabic commonly spoken across the Arab world, but lack standard orthographies. |
| Approach: | They present a corpus of 10,000 sentences from five Arabic city dialects represented in the Conventional Orthography for Dialectal Arabic (CODA) they use a bootstrapping technique to speed up annotation and compare similarity between dialects before and after CODA annotation. |
| Outcome: | The proposed method speeds up the annotation process and shows similarity between the dialects before and after CODA annotation. |
Machine Translation of Low-Resource Spoken Dialects: Strategies for Normalizing Swiss German (L18-1)
Copied to clipboard
| Challenge: | Using character-based neural MT, we normalize Swiss German input to address regional diversity. |
| Approach: | They propose to use character-based neural MT to normalize Swiss German input and phrase-based statistical MT for a low-resource family of dialects. |
| Outcome: | The proposed system achieves 36% BLEU score when translating from the Bernese dialect. |
What is lost in Normalization? Exploring Pitfalls in Multilingual ASR Model Evaluations (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing text normalization routines that target Indic scripts are flawed when applied to multilingual automatic speech recognition models. |
| Approach: | They propose to develop text normalization routines that leverage native linguistic expertise to ensure more robust and accurate evaluations of multilingual automatic speech recognition models. |
| Outcome: | The proposed normalization routines can be leveraged to improve performance metrics for Indic languages. |
Burmese Speech Corpus, Finite-State Text Normalization and Pronunciation Grammars with an Application to Text-to-Speech (2020.lrec-1)
Copied to clipboard
Yin May Oo, Theeraphol Wattanavekin, Chenfang Li, Pasindu De Silva, Supheakmungkol Sarin, Knot Pipatsrisawat, Martin Jansche, Oddur Kjartansson, Alexander Gutkin
| Challenge: | Using crowd-sourced speech corpus and finite-state transducer grammars, we build a text-to-speech system for Burmese, a tonal Southeast Asian language from the Sino-Tibetan family. |
| Approach: | They propose an open-source crowd-sourced multi-speaker speech corpus and finite-state grammars for performing grapheme-to-phoneme conversion for Burmese. |
| Outcome: | The proposed system performs well for Burmese in a low-resource setting. |
NeoAraBERT: A Modern Foundation Model for Arabic Embeddings with Diacritics-Aware Tokenization and POS-Targeted Masking (2026.findings-acl)
Copied to clipboard
Chadi Abou Chakra, Hadi Khaled Hamoud, Osama Rakan Al Mraikhat, Qusai Abu Obaida, Mohamad Ballout, Fadi Zaraket
| Challenge: | NeoAraBERT is an open-source text-embedding model for Arabic. |
| Approach: | They propose to train Arabic text-embedding models on open-source datasets . they benchmarked NeoAraBERT against five top-performing Arabic models on 23 tasks . |
| Outcome: | The proposed model outperforms five other models on 23 tasks in Arabic . it shows substantial improvement on classical and modern standard Arabic compared to other models . |